Margin-Based Ranking Meets Boosting in the Middle

We present a margin-based bound for ranking in a general setting, using the L∞ covering number of the hypothesis space as our complexity measure. Our bound suggests that ranking algorithms that maximize the ranking margin will generalize well. We produce a Smooth Margin Ranking algorithm, which is a modification of RankBoost analogous to Approximate Coordinate Ascent Boosting. We prove that this algorithm makes progress with respect to the ranking margin at every iteration and converges to a maximum margin solution. In the special case of bipartite ranking, the objective function of RankBoost is related to an exponentiated version of the AUC. In the empirical studies of Cortes and Mohri, and Caruana and Niculescu-Mizil, it has been observed that AdaBoost tends to maximize the AUC. In this paper, we give natural conditions such that AdaBoost maximizes the exponentiated loss associated with the AUC, i.e., conditions when AdaBoost and RankBoost will produce the same result, explaining the empirical observations.